Paper trade improvements #462

algorandpa · 2023-02-15T00:18:56Z

Added ledger and positioning support for paper mode
Ensure we immediately write to ledger and update position in real time
Added local txn state in Paperboi

Test suite todo:

clear a paper order, ensure the pp shows up in pps.toml, stop and start pikerd, ensure the pp is relayed on open_ems() connect
ensure after a return to a net zero position, the pps.toml does not get a leaked account entry as per comment Paper trade improvements #462 (comment)
when loading a paper position after a full restart of the service stack (i.e. shutting down and restarting pikerd) if you make a new clear to begin "closing out" (i.e. reducing the dst asset) the paper engine is not sending the correct BrokerdPosition state as per the comment Paper trade improvements #462 (comment), as such we need to add test code to process the trade_stream: tractor.MsgStream msgs and ensure the eventual last position msg matches the state in the pps.toml

Future dark-order and alert storage todo test suite (maybe to do in a follow up PR):

~~submit an order that does not clear, stop and start pikerd, ensure order is re-registered in emsd~~ => deferred to #463
~~will likely requires Order serialization to file system and load~~ => deferred to #464

goodboy · 2023-02-16T19:21:07Z

@algorandpa while i'm testing and reviewing this you might as well make your follow issue(s) for that last bullet and whatever else 🏄🏼

algorandpa · 2023-02-16T20:01:59Z

Issue created for second task here @goodboy

pytest.ini

piker/pp.py

piker/clearing/_client.py

piker/clearing/_ems.py

piker/clearing/_paper_engine.py

piker/config.py

piker/data/feed.py

goodboy · 2023-02-16T22:42:33Z

piker/pp.py

 )
+from typing import Generator


duplicate import of that Generator no?

piker/pp.py

tests/test_paper.py

goodboy · 2023-02-28T18:48:52Z

Though i'm sure @guilledk is totally aware of the details, just putting this for myself and the knowledge of others as per the fix landing in #470.

Turns out the msging layer seems to be completely fine in terms of zeroing pps despite using floats throughout sizing arithmetic; the issue seems to be purely in our .pp.PpTable size storage and arithmetic sub-sys.

As an example here i was able to get the precision error to linger in the pps.toml but in the UI and at the msging layer there was a zeroed pp every time i started a new session and the accum_size = 2.7755575615628914e-17 was present in the clearing table:

[binance.paper."btcusdt.binance"]
size = 2.7755575615628914e-17
ppu = 23489.324999999997
bsuid = "btcusdt"
clears = [
 { dt = "2023-02-28T18:36:42.360930+00:00", ppu = 23455.5, accum_size = -0.04264, price = 23455.5, size = -0.04264, cost = 0, tid = "34323ca0-277b-4f9f-b77e-1ef4718c49c5" },
 { dt = "2023-02-28T18:36:44.500479+00:00", ppu = 23456.435, accum_size = -0.08528, price = 23457.37, size = -0.04264, cost = 0, tid = "9996508c-b94b-489f-910f-c9360c9ce031" },
 { dt = "2023-02-28T18:36:46.400183+00:00", ppu = 23456.00666666667, accum_size = -0.12791999999999998, price = 23455.15, size = -0.04264, cost = 0, tid = "15dfe435-d54b-450d-b1ec-21db6e535c17" },
 { dt = "2023-02-28T18:38:37.989248+00:00", ppu = 23462.75, accum_size = 2.7755575615628914e-17, price = 23462.75, size = 0.12792, cost = 0, tid = "138dcf2b-5ab9-43b3-bb45-0465da730ac9" },
 { dt = "2023-02-28T18:44:02.788944+00:00", ppu = 23480.89, accum_size = 0.04258000000000003, price = 23480.89, size = 0.04258, cost = 0, tid = "e098b4d2-72b2-41a9-b1f4-e74f6c457c56" },
 { dt = "2023-02-28T18:44:11.890799+00:00", ppu = 23480.89, accum_size = 2.7755575615628914e-17, price = 23482.39, size = -0.04258, cost = 0, tid = "672167d1-ee78-4e54-91a6-7fc63f96349f" },
 { dt = "2023-02-28T18:44:41.429072+00:00", ppu = 23491.16, accum_size = 0.042570000000000024, price = 23491.16, size = 0.04257, cost = 0, tid = "df3eeead-e6ad-477a-9720-c5143ace03cf" },
 { dt = "2023-02-28T18:44:44.812730+00:00", ppu = 23489.324999999997, accum_size = 0.08514000000000002, price = 23487.49, size = 0.04257, cost = 0, tid = "30f471e6-e1c0-484b-91cd-cd9c9aec8804" },
 { dt = "2023-02-28T18:44:48.019707+00:00", ppu = 23489.324999999997, accum_size = 0.042570000000000024, price = 23488.71, size = -0.04257, cost = 0, tid = "6f6f2a34-3fe0-436b-9896-4874172cbfc2" },
 { dt = "2023-02-28T18:45:31.141610+00:00", ppu = 23489.324999999997, accum_size = 2.7755575615628914e-17, price = 23480.55, size = -0.04257, cost = 0, tid = "46440830-caed-4b76-b6b3-0ac27fa7b054" },
]

…arketstore

remove duplicate import

format function calls

Add extra line in conftest

goodboy

Lookin good enuf to me!

Nice work; super stoked to have a suite to start hardening our clearing layer and refining our order ctl APIs 🏄🏼

Land when y'all are ready.

algorandpa requested review from goodboy and guilledk February 15, 2023 00:18

algorandpa force-pushed the paper_trade_improvements_rebase branch from c445ccb to e62870c Compare February 16, 2023 00:12

algorandpa changed the title ~~Paper trade improvements rebase~~ Paper trade improvements Feb 16, 2023